Show navigation

Batching and routing in ShareScan

Batching and routing is the ShareScan capability to split a scan job into separate batches and handle/process/route the output documents independently of one another.

For instance, you can scan a stack of sheets and have a separate document created out of every ten-page batch. If you scan, say 40 pages, you can have 4 output documents created out of this single scan job. For every document, you can specify file format, connector target, etc.

All ShareScan connectors support batching.

All connectors support batching in a scenario when either

• there is no capture user interface shown on the MFP screen (That is, MFP end-users do not need to set file format, document metadata: they simply press the button of a connector and can leave the device as soon as all sheets have been scanned), or
• when capture user interface is shown on the MFP screen, but Background processing is enabled. Background processing means that MFP end-users do not have to wait at the device until the output documents are created. Processing is taken over by the ShareScan Manager.

There are two basic types of batching:

• connector-level batching (also known as 'native' batching): when document (meta)data (file naming, format, author, subject, etc. - depending on the actual connector) MFP screens are shown only once, and user choices are applied to all output documents for that scan job. (Using the example above: you choose PDF file format once, when your first 10-page batch is scanned, and all 4 documents created out of your scan job will be PDFs.)
• global batching (also known as 'automatic/generic' batching): lets you set document data for each output document in the scan job. You can select PDF file format for your first 2 documents, then pick DOC for your third one, and PDF again for your last 10 pages.

What is Batching?

Batching or batch processing involves the automatic execution of scanning jobs on MFP devices with the help of pre-written scripts/algorithms without the need for human or any other form of intervention. If you have a large stack of hard copy documents to scan, you can easily split them into several smaller batches according to your different scanning needs. You can direct these diverse scanning tasks to multiple devices/destinations and they will be completed without interruption.

Major version batching differences

There is a difference in the way ShareScan handles batching scenarios in pre-5.2 versions and in version 5.2 and higher.

Pre-5.2 version batching

Versions prior to 5.2 implemented so-called 'native' batching: all pages/documents and batching data are sent to the connector in one pass, and the connector is responsible to process every document selecting the proper batching data. The procedure is the same in both Online and Background mode.

Not all connectors have native batching capabilities - and some are capable of this when in Express mode (no end-user interaction is required at the MFP).

Version 5.2 batching

To make batching available for connectors without this capability and to eliminate the need to modify existing 3^rd party connectors, ShareScan 5.2 offers 'automatic' batching.

This model is driven by a Document Service assigned to the connector profile: every document created from the scanned pages is sent sequentially to the connector with the proper routing and indexing data belonging to the document. The connector behaves exactly the same way as it was used with a simple scanning job with the pages of the 1^st document, then the 2^nd etc. This also means that if configured so (non-Express mode), all connector forms are displayed on the device. But generic forms or those belonging to the document services are not displayed again before sending the 2^nd, 3^rd etc. document to the connector.

Connector functionalities

Original connector functionalities are redesigned since ShareScan 5.4 to make their usage more comfortable.

Connectors can be grouped based on how their functionalities are changed:

CORRECTED original functionality:

EXTENDED original functionality:

Native vs automatic batching

To help you choose which batching behavior to use, consider the following:

• Not all connectors have native batching capabilities. Native batching is only available for a limited set of connectors (for some, in Express mode only) - for details, see the summary table below. Native batching support (in Express mode) is introduced by ShareScan 5.2 for the following connectors: Exchange Mail, Exchange Fax, SMTP via LDAP, SMTP Fax and Worksite.
• Less time spent at the device. In case of native batching, any routing, storing, or authentication-related tasks are solely based on the values arriving from Data Publishing. Thus, once properly configured, there is less need for user interaction, so the overall time spent at the MFP side decreases.
• Automatic batching covers more connectors, even the ones incapable of native batching.
• Automatic batching gives you more MFP-side interaction control over your batches. This practically means that end-users interact with routing and indexing-related MFP screens for each document in the batch.
• Automatic batching is a better choice if the output documents in your batch are not more than a couple of pages and you have the proper infrastructure to facilitate quick processing.

Native batch processing support depends on two connector capabilities:

These capabilities are described as service settings for every connector in the <connector name>Services.xml files located under the connector data folder.

In this file the supports-NativeBatching setting specifies the SupportsNativeBatchingInExpressMode capability and the supports-NativeBatchingInUIMode setting specifies the SupportsNativeBatchingInUIMode capability.

Example: (Worksite connector - supports native batching in Express mode only):

file name: WorksiteConnectorServices.xml

file location C:\%programdata%\Kofax\ShareScan\Connectors\WorksiteConnector\Data

Settings:

• supports-NativeBatching="true"
• supports-NativeBatchingInUIMode="false"

If these settings are missing from this file, then ShareScan behaves as if the connector did not support native batch processing at all. (This is the case with 3^rd party connectors.)

If the connector profile has no Data Published value coming from the scanned document, ShareScan selects native batch behavior. ShareScan does not display an error message on whether the connector supports native batching or not. As this is not a document dependent batch situation, the Data Published value is the same for every document in the batch. This can occur when a document service Published key is not mapped to a connector Published key in the connector profile (for example, if the connector uses only Client and Matter Published keys coming from Cost Recovery system).

Note: 

Do not modify these factory connector settings.

These settings are described in this file so that 3^rd party connector batch processing behavior could be modified simply on connector level.

You can also modify connector batch processing behavior by configuring new advanced settings related to batch processing behavior (see later).

Automatic batching and background processing

If you are using automatic batching, connector behavior depends on the Background processing setting.

If Background processing is enabled the connector behaves as if the user had clicked the Done button on the last screen and had scanned the pages of the next document with the same settings.

If Background processing is disabled the connector behaves as if the user had clicked the New document button on the last screen and had scanned the pages of the next document with the same settings.

Selection of batch processing behavior

ShareScan is able to determine if batching is needed: if a document service (assigned to the given connector profile) uses document splitting, the ShareScan system treats this as such a case.

ShareScan automatically selects either native or automatic batch behavior. Unsupported configurations display an error message.

Based on connector batch supporting capabilities, configuration and other settings, it automatically selects the batch processing behavior to be applied.

*If there are no connector DP keys mapped to the DP keys of a document service assigned to the connector profile, the Manager executes native batching to preserve compatibility.

In order to use batching with connectors, you do not need to perform any specific actions during connector profile Data Publishing mapping configuration. The Data Publishing can be configured in the regular way, and when the batching is available from the Document Service side, and the specific connector allows for batch processing, the connector processes data in batched manner.

Note:

Be aware that when batch processing is executed, exceptions do not stop processing, but they are traced, and an error report is included into the notification message of the Notification service, if Notification is configured. For more information on the Notification service, click here.

Connector capabilities related to batch processing

Batch processing scenarios

The following tables summarize automatic batch processing behavior selections based on connector capabilities and default capability settings.

Background processing DISABLED

Default capability settings:

Background processing ENABLED

Default capability settings:

Advanced settings for batch processing

There are two advanced settings which enable ShareScan Administrators to select the batch processing behavior (automatic or native batching) for a connector profile. Use these settings to support batch scenarios by default unsupported by ShareScan (mostly 3^rd party connectors).

1. ConnectorsAllowedToSelectBatchBehaviorInBackgroundMode

The value of this setting is a list of connector IDs where batch behavior can be selected if Background processing is enabled.

2. ConnectorsAllowedToSelectBatchBehaviorInOnlineMode

The value of this setting is a list of connector IDs where batch behavior can be selected if Background processing is disabled.

If a connector ID is in the list, then you can select batch behavior at the profiles of this connector.

Connector IDs are stored in the registry under the following key:

HKEY_CLASSES_ROOT\eCopyShareScan\Connectors\<ConnectorID>.MainControl\ConnectorID

If the connector ID appears in one of these settings' value, then an Automatic batching checkbox is displayed under the Services group on Settings pane in the Administration Console.

This checkbox is displayed only if a batch processing is configured (a document service is assigned to the connector profile configured to split the scanned pages into more documents).

If this Automatic batching checkbox is ticked, then Automatic batching is used, otherwise Native batching is used. If the ConnectorsAllowedToSelectBatchBehaviorInBackgroundMode value contains the connector ID, then the checkbox is displayed if Background processing is enabled at the profile. If the ConnectorsAllowedToSelectBatchBehaviorInOnlineMode value contains the connector ID, then the checkbox is displayed only if Background processing is disabled at the profile.

PublishingKeysForAllBatches Registry Setting

If a Data Publishing value is set before scanned pages are split into more documents, it is applied only to the first document.

The rest of the documents will have the default value of the Data Publishing key.

This is the case when a Database Lookup Extender profile is processed before the document service/extender profile configured with split option or with connector-based splitting (i.e. splitting by Number of Pages, File Size, Blank Page).

The new PublishingKeysForAllBatches registry setting makes it possible for these types of Data Publishing values to be applied to all batch documents.

To achieve this, the names of these Data Publishing keys must be specified (separated by semicolon) as a value of this registry setting.

Key = HKLM\Software\{Wow6432Node\}Kofax\ShareScan\ShareScanManager\

Name = PublishingKeysForAllBatches

Data = Fully qualified source DP key names separated by semicolon

Example:

LookupDocumentService.default.GEO

where LookupDocumentService is the name of the extender, default is the name of the profile and GEO is the Data Publishing key name for that profile.

For all Data Publishing source keys listed in this setting, the connector uses the Data Publishing source key value of the first document for all documents when they are mapped to a connector Data Publishing key.

Connector Configurations and Example Scenarios

Worksite

The Worksite connector can support batching for those profile configurations which do not require interaction from the end-user. Thus, you have to have a single destination set, or you must have folder navigation disabled (meaning no navigation screens are displayed on the MFP screen). For properly routing the processed documents, you must also provide the relevant credentials, so Worksite can perform the tasks without having to prompt for user interaction.

To achieve a fully automated batching and routing in your workflow, the following configuration settings are required:

1. Ensure that the Worksite connector profile contains only a single destination.
2. Select the single destination, and click Edit.
3. On the Navigation tab of the Edit Destinations pane, set Auto-select or Specify mode.
4. On the Navigation tab of the Edit Destinations pane, ensure that the Provide a storage confirmation screen option is UNCHECKED.
5. On the Navigation tab of the Edit Destinations pane, ensure that the Provide autonavigation property screen as needed option is UNCHECKED.
6. On the Attributes tab of the Edit Destinations pane, the document profile attribute population method must be set to Auto-index.

When all the above requirements are met, the Worksite connector supports batching for all of the published keys.

For more information on Worksite connector customization, click here.

SharePoint

The following configuration settings are required for batching support:

• Set the Show option of the Columns tab to Auto-Index.
• The Retrieve Content Type from Data Publishing option is configured on the Columns tab
• You must set at least one column as Published (a highlighted key must be displayed before the column name).

If there is no column configured as a Published column, the Column Information form is displayed independently from the batching mode and the column values specified by the user are set to all documents in the batch.

Auto-select location is configured

In this case the location where a document has to be stored can come from data publishing.

Documents in the batch can be stored to different locations depending on the batched published value without any user interaction. The Navigation form is not displayed.

If the ColumnInformation is not displayed in this case, the document is stored with content type and column values specified on SharePoint server within Administration Console. Otherwise they are retrieved from data publishing depending on Retrieve Content Type from Data Publishing setting, Auto-index show option and column settings.

Retrieve Content Type from Data Publishing

In this case the content type of the document to be stored can come from data publishing. Documents in the batch can be stored with different content type (and so different column values) depending on the batched published value without any user interaction. The ColumnInformation form is not displayed so the user has no possibility to set any column values on the device even if All, Required or Auto-Index show option is configured.

Auto-Index show option is configured with at least one column as Published column

In this case, the connector attempts to use batched published data in those columns that are configured as Published columns (signaled by the highlighted key icon before the column in the column list) in the automatically selected runtime content type.

The runtime content type is

• the content type retrieved from data publishing, if Retrieved Content Type from Data Publishing setting is also configured and the content type retrieved from Data Publishing exists at the runtime location
• the default content type of the runtime location, if Retrieved Content Type from Data Publishing setting is also configured and the content type retrieved from Data Publishing does not exist at the runtime location
• the default content type of the runtime location, if Retrieved Content Type from Data Publishing setting is not configured

Runtime location is

• the location retrieved from data publishing,
• the location selected by the user on Navigation form, or
• the location specified as Hyperlink in Admin Console if Navigation was not enabled at the profile.

In this case the ColumnInformation form is not displayed and the connector tries to store the document with the proper batched published column values.

The ColumnInformation form is displayed in batching mode only if

• there is no column configured as Published column,
• Retrieve Content Type from Data Publishing is not configured and
• Auto-select location is not configured.

In this case the column values specified by the user on ColumnInformation form will be set to all documents in the batch as in the earlier versions of the connector.

Using SharePoint batching and routing - example scenario

The following procedure outlines an example for using the SharePoint connector in connection with Business Automation, Image Control, and Data Publishing to automatically route and store documents to specific SharePoint locations using information gained from the barcodes located on the scanned pages.

1. Ensure that you have Microsoft SharePoint installed.
2. Start the ShareScan Administration Console.
3. Go to the Services pane, select Image Control, and click the Page Templates tab. On the Template tab of the Template Settings dialog, create a new template for the process.
   This allows you to specify which barcodes to use, and the location of the barcodes on the scanned pages.
4. On the Barcodes tab of the Template Settings dialog, set the barcode type to the type you are planning to use, and add a delimiter character for separating multiple documents.
   Click OK, and save the service profile.
5. Go to the Services pane, select Image Control, and click the Document Splitting tab, disable the barcode page removal, and check the Split at page option.
   
6. Go to the Connectors pane, select SharePoint, and click New to create a new destination.
   On the Authentication tab, the location under Hyperlink will be handled as the top level of the storage location.
7. On the Navigation tab, set the Auto-select location option. This ensures that the location information can be obtained from Data Publishing automatically.
   Note that the tooltip displays the storage key name if you hover the cursor over the Auto-select location icon.
8. On the Columns tab, set the connector to get the relevant data from Data Publishing by setting the Show field to Auto-index, and ensuring that the Retrieve Content Type from Data Publishing checkbox is enabled.
   Do this for all locations and document types you plan to use. (Hovering the cursor over a key displays its name.)
9. Save your connector configuration. At this point, the SharePoint connector is ready and configured to be used; you can move on to configuring Data Publishing to allow taking these metadata into use.
10. On the Advanced tab of the Administration Console, select Tools > Data Publishing Mapping Tool.
11. Use the Filter > Published Keys function to select the service profile you created in steps 3-4.
    Repeat the same using Filter > Connector Keys, and select the keys you created during the configuration of the SharePoint connector. Be advised that you have to do it one at a time, thus if you have 5 keys, you have to perform the filtering and adding five times.
    Note that due to the nature of the SharePoint software, the key names are NOT the column names displayed in the SharePoint connector configuration grid, but the SharePoint internal names.
12. Go to the Services pane, and select Notification. Enter the proper credentials for your system configuration. Then save your service profile.
13. On the Connector pane, select the SharePoint connector, and use the Settings pane on the right-hand side to ensure that the Notification and Image Control services are Enabled for the connector profile.
    The Image Control is mandatory, as otherwise no barcode recognition is possible. The Notification service provides you with detailed feedback information in case any issues occur during the scan process.
14. You can begin scanning. Your scanned documents will be stored to the SharePoint location under the proper folder, with the proper content type.